Probability density estimation using artificial neural networks

We present an approach for the estimation of probability density functions (pdf) given a set of observations. It is based on the use of feedforward multilayer neural networks with sigmoid hidden units. The particular characteristic of the method is that the output of the network is not a pdf, therefore, the computation of the network’s integral is required. When this integral cannot be performed analytically, one is forced to resort to numerical integration techniques. It turns out that this is quite tricky when coupled with subsequent training procedures. Several modifications of the original approach (Modha and Fainman, 1994) are proposed, most of them related to the numerical treatment of the integral and the employment of a preprocessing phase where the network parameters are initialized using supervised training. Experimental results using several test problems indicate that the proposed method is very effective and in most cases superior to the method of Gaussian mixtures.  2001 Elsevier Science B.V. All rights reserved. PACS: 02.70.Lq; 07.05.Kf; 07.05.Mh